Segmentation of tissue images using color and texture

ABSTRACT

The present techniques provide for the processing of color tissue images based on image segmentation. In an exemplary embodiment, the color and texture features of pixels in a tissue image are used to generate a matrix of feature vectors. A subset of feature vectors is selected from the matrix of feature vectors and a set of colors and textures are derived using the tissue image and the subset of feature vectors. An initial segmented tissue image is then generated from this set of colors and textures.

BACKGROUND

The invention relates generally to the processing of color images, whichmay be used to assess tissue abnormality in a tissue image. Inparticular, the present techniques relate to the segmentation of colortissue images.

Digital microscopy has become increasingly important in pathology andmorphology. Images of stained tissue slides may be obtained and used bypathologists to recognize abnormal tissue structures associated withcancer. For example, a prostate cancer diagnosis is typicallyestablished by histopathology using hematoxylin and eosin (H&E) stainedtissue sections, which are evaluated by a pathologist to subjectivelyassess the cancer state or grade. A pathologist's assessment of thecancer stage may be based upon gland and nuclei distributions andmorphological features observed in an image of the cancerous tissue andhow these distributions and features differ from those of a normaltissue image. However, human pattern recognition may be time consumingand inefficient because of the number of new cancer cases each year andthe limited resources, such as the number of pathologists available.

To improve throughput, tissue microarrays (TMA) may be used forpathology research. In this approach, tissue cores from differentpatients are embedded in a paraffin block and sliced to give multipleregistered arrays. These multiple tissue cores are simultaneouslyprocessed to remove staining variability and to reduce labor. However,even after staining variability is removed, an accurate and efficientevaluation may still require segmentation of features of interest in thetissue slides. Image segmentation may generally involve splitting animage into several components and assigning each pixel in the image to arespective component. Specifically, segmentation may be useful forclassifying tissue image elements, such as pixels or larger imagestructures, into useful groups or categories.

Manual segmentation of a tissue image may be extremely time intensive.Moreover, the manual analysis should be done by an expert pathologist,whose time is limited and valuable. Additionally, automated segmentationmethods often result in inaccurately segmented images. In particular,current techniques for automated segmentation of images are oftenunsuitable for use on color images, such as stained images of tissue,due to the interdependence of the color components of each pixel. Inaddition, such automated segmentation techniques may be computationallyintensive. Therefore, a more efficient image segmenting process fortissue images is desired.

BRIEF DESCRIPTION

In accordance with an exemplary embodiment of the present technique, atissue image is analyzed to determine color and texture components foreach pixel in the image. Each pixel of the image then has a vector offeature values that undergo a two-part self organized mapping (SOM) toproduce a segmented image. The two-part SOM may be referred to as ahierarchical SOM, or HSOM. The first part of the HSOM selects a pixelfeature vector and updates the feature vector of a closely matchingpixel or computational unit of a second image or computational grid toresemble the feature vector of the selected pixel. The process isiterative through different pixel segments until all the pixel segmentsin the second image or computational grid have been updated. Theseupdated values represent the dominant color and texture characteristicsin the tissue image. The second part of the HSOM improves the dominantfeatures further by learning from the dominant characteristics producedby the first part. The original tissue image, when processed using thetwo-part HSOM, yields a cohesively segmented image as the values of thetissue image pixels are updated to match the values of the previouslyupdated pixels of the second image or computational grid. One advantageof using the HSOM is that each high dimensional data vector is mapped toa low-dimensional discrete value so that comparing the values implicitlycontains comparison of the original distances. In one embodiment,similar color regions in the segmented image generated by the HSOMprocess are merged to create a segmented image with improved homogeneitywithin the regions.

In exemplary embodiments, this process needs no training set and no apriori knowledge (though such a priori knowledge may be used ifavailable), as the only information needed for unsupervised clusteringis the information within the image itself. Furthermore, this processalso utilizes texture analysis to more clearly differentiate betweenuseful tissue structures, and is robust in the presence of incompletedata. The resulting segmented image using the present technique is anaccurately and clearly segmented image with no processing supervision.

DRAWINGS

These and other features, aspects, and advantages of the presentinvention will become better understood when the following detaileddescription is read with reference to the accompanying drawings in whichlike characters represent like parts throughout the drawings, wherein:

FIG. 1 is a diagrammatical view of an exemplary system for use insegmenting tissue images, in accordance with aspects of the presenttechnique;

FIG. 2 is a flow chart depicting acts for segmenting a tissue image, inaccordance with the present technique;

FIG. 5 is a flow chart of a training stage in a competitive learningmethod, in accordance with the present technique;

FIG. 6 is a flow chart depicting acts for segmenting an image using atrained weight output, in accordance with the present technique;

FIG. 7 is a flow chart depicting acts for region merging using athreshold standard, in accordance with the present technique; and

FIG. 8 is a flow chart depicting acts for region merging using a regionnumber standard, in accordance with the present technique;

DETAILED DESCRIPTION

Embodiments of the present technique are generally directed to theunsupervised clustering and region merging of a color tissue image usingcolor and texture features, resulting in the segmentation of usefultissue structures. This method leverages the advantages of unsupervisedclustering, along with the use of multi-spectral features includingtexture. Being unsupervised means that the present technique requires noa priori knowledge (though such a priori knowledge may be used ifavailable) about the image being segmented. It uses information withinthe given image to identify dominant clusters and does not require anextensive set of training images. Additionally, in embodiments employingboth color and texture, the use of texture features along with colorfeatures better enables the present technique to segment differenttissue classes. While previous methods may have inaccurately classifiedspectrally similar tissue types, the use of texture features enables thepresent techniques to better separate useful tissue structures.

For example, in an exemplary implementation, the color and texturevalues of each pixel form the pixel's feature vector, and the featurevectors for all the pixels in the image form a feature matrix that isthe input for unsupervised clustering. One embodiment of the presenttechnique uses a two-stage hierarchical self-organizing map (HSOM) toprovide a framework for unsupervised clustering. In one suchimplementation, the first stage of the technique employs a fixed-sizetwo-dimensional map that captures the dominant color and texturefeatures of an image in an unsupervised mode. The second stage combinesa fixed-size one-dimensional feature map and color merging, to controlthe number of color clusters formed as a result of the segmentation. Insuch an embodiment, the HSOM may be seen as a pixel classifier in that achosen subset of pixels from the image may be used to “learn” theunderlying clusters, thereby obviating the need for a separate set oftraining images. Such a subset of pixels may either be selected based onsub-image statistics or in a sequential or random manner from the entireimage. Though it may be difficult for any training set to capturevariation present across all combinations of tissue and pathology, thepresent technique is advantageous in that it captures the variation inindividual images and does not require a set of training images.

In one exemplary embodiment, the present technique may be implementedusing an artificial neural network (ANN) or other competitive learningmethod. In such an ANN, the image component (such as a pixel or group ofpixels) having an associated feature vector may be represented by acomputational element, such as a neuron. In certain embodiments of thepresent technique, more than one neuron (or corresponding computationalelement) is allowed to learn each iteration, i.e., the winning neuronand those neurons determined to be in the neighborhood of the winningneuron may all participate in the learning process, leading to anordered feature-mapping. Such an embodiment is in contrast to othercompetitive learning techniques where only the winning neuron is allowedto learn.

The segmentation of the image into tissue structures enablesquantitative analysis and evaluation of the tissue disease state. Forexample, this can be done by extracting discriminate features from thesegmentations that can be correlated with tumor staging scores (such asGleason scores), which may lead to diagnostic information. A systemincorporating such measures can be used as a second reader in apathology examination. For example, once the pathologist has reviewedand scored the tissue, the pathologist's results can be compared withthe automatic scoring to give further confidence to manual scoring.Further, in cases where the manual and automatic scores differ, theinformation from the automatic score can be used to give a potentiallybetter score. The ability of such an automatic system to point out areasof the tissue that are potentially problematic can aid the pathologistto focus on areas that might have been missed.

Additionally, such a system can also be used to train new pathologistsby showing a large number of images automatically quantified. Since newpathologists may require a large number of images during their trainingprocess, a large database of manually segmented images may be difficultto obtain. Thus, the unsupervised segmentation of images describedherein may be useful for providing training images or establishingtraining databases.

With the foregoing in mind, an exemplary tissue image segmentationsystem 10 capable of performing the present technique is depicted inFIG. 1. Generally, the imaging system 10 includes an imager 12 thatdetects signals and converts the signals to data that may be processedby downstream processors. As described more fully below, the imager 12may operate in accordance with various physical principles, such asoptical principles, for creating the image data. In general, the imager12 generates image data, of any dimension, in a conventional medium,such as photographic film, or in a digital medium. Furthermore, in oneembodiment, the imager 12 may provide some degree of magnification whilein other embodiments the imager 12 provides little or no magnification.For example, in one implementation, the imager 12 may be a microscope,such as a high-throughput microscope, suitable for image and/or videoacquisition under magnification at suitable light wavelengths (such asvisible, infrared, and/or ultraviolet light wavelengths). For instance,the imager 12 may be any suitable imaging device, including afluorescence microscope, a confocal fluorescence microscope, a laserscanning confocal microscope, or a total internal reflectionfluorescence microscope.

In the depicted embodiment, the imager 12 is configured to image atissue section. The tissue section may be stained with hematoxylin andeosin (H&E) or some other stain or combination of stains suitable forviewing tissue sections. For example, in an exemplary implementation thetissue section is provided as a tissue microarray (TMA) 14, which isoften used for high-throughput pathology research where multiple tissuesare simultaneously processed to remove staining variability.

In one embodiment, the imager 12 operates under the control of systemcontrol circuitry 22. The system control circuitry 22 may include a widerange of circuits, such as circuitry controlling the emission of varioustypes of electromagnetic radiation (such as visible, infrared, and/orultraviolet light, X-rays, electron beams, and so forth) for use in theimaging process. Likewise, in some embodiments, the system controlcircuitry 22 may include timing circuitry, circuitry for coordinatingdata acquisition in conjunction with movement of a sample, circuitry forcontrolling the position of the imager 12 and/or the TMA 14, and soforth.

In the present context, the imaging system 10 may also include memoryelements 24, such as magnetic or optical storage media, for storingprograms and routines executed by the system control circuitry 22 and/orby associated components of the system 10, such as data acquisitioncircuitry 26 and/or data processing circuitry 28. The stored programs orroutines may include programs or routines for performing all or part ofthe present technique.

In the depicted embodiment, data acquisition circuitry 26 is employed toacquire image data from the imager 12. In optical embodiments, the dataacquisition circuitry 26 may be configured to acquire image data via oneor more optical sensing elements, such as may be found in digitalcameras, that are disposed on or in the imager 12 or in opticalcommunication with the imager 12. The acquired image data may be digitalor analog in nature. In embodiments where the initially acquired imagedata is analog in nature, the data acquisition circuitry 26 may also beconfigured to convert the analog data to a digital format. Likewise, thedata acquisition circuitry 26 may be configured to provide some initialprocessing of the acquired image data, such as adjustment of digitaldynamic ranges, smoothing or sharpening of data, as well as compiling ofdata streams and files, where desired.

The image data acquired by the data acquisition circuitry 26 may beprocessed, such as by data processing circuitry 28 in the depictedembodiment. For example, in certain embodiments, the data processingcircuitry 28 may perform various transformations or analyses of theimage data, such as ordering, sharpening, smoothing, featurerecognition, and so forth. Prior or subsequent to processing, the imagedata may be stored, such as in memory elements 24 or a remote device,such as a picture archiving communication systems or workstationconnected to the imaging system 10, such as via a wired or wirelessnetwork connection.

The raw or processed image data may, in some embodiments, be provided toor displayed on an operator workstation 32. In such embodiments, theoperator workstation 32 may be configured to allow an operator tocontrol and/or monitor the above-described operations and functions ofthe imaging system 10, such as via an interface with the system controlcircuitry 22. The operator workstation 32 may be provided as a generalpurpose or application specific computer 34. In addition to a processor,the computer 34 may also include various memory and/or storagecomponents including magnetic and optical mass storage devices, internalmemory, such as RAM chips. The memory and/or storage components may beused for storing programs and routines for performing the techniquesdescribed herein that are executed by the computer 34 or by associatedcomponents of the imaging system 10. Alternatively, the programs androutines may be stored on a computer accessible storage and/or memoryremote from the computer 34 but accessible by network and/orcommunication interfaces present on the compute 34.

The computer 34 of the operator workstation 32 may also comprise variousinput/output (I/O) interfaces, as well as various network orcommunication interfaces. The various I/O interfaces may allowcommunication with user interface devices of the operator workstation32, such as a display 36, keyboard 38, mouse 40, and/or printer 42, thatmay be used for viewing and inputting configuration information and/orfor operating the imaging system 10. The various network andcommunication interfaces may allow connection to both local and widearea intranets and storage networks as well as the Internet. The variousI/O and communication interfaces may utilize wires, lines, or suitablewireless interfaces, as appropriate or desired.

Though a single operator workstation 32 is depicted for simplicity, theimaging system 10 may actually be in communication with more than onesuch operator workstation 32. For example, an imaging scanner or stationmay include an operator workstation 32 used for regulating theparameters involved in the image data acquisition procedure, whereas adifferent operator workstation 32 may be provided for viewing andevaluating results.

For the purpose of explanation, certain functions and aspects of thepresent technique have been described as being separate and distinct oras being associated with certain structures or circuitry. However, suchdistinctions have been made strictly to simplify explanation and shouldnot be viewed as limiting. For example, for simplicity, the precedingdiscussion describes implementation via a discrete imaging system 10 andoperator workstation 32. As will be appreciated, however, certainfunctions described as being performed by the imaging system 10, such asdata acquisition, data processing, system control, and so forth, mayinstead be performed on the operator workstation 32 or may havediffering aspects, some of which are performed on the imaging system 10and others of which are performed on the operator workstation 32.Indeed, in practice, virtually all functions attributed to the imagingsystem 10, with the possible exception of the functions attributed tothe imager 12, may be performed on an operator workstation 32. In otherwords, the data acquisition circuitry 26, memory 24, data processingcircuitry 28, and/or system control circuitry 22 may be provided ashardware or firmware provided in an operator workstation 32 and/or assoftware executable by the operator workstation 32. For example, some orall of the circuitry described herein may be provided as routinesexecuted on a suitable processor or coprocessor of a computer 34 of anoperator workstation 32. Indeed, it should be understood that the termcircuitry, as used herein, encompasses dedicated or generalized hardwareor firmware implementations and/or processor-executable softwareimplementations suitable for implementing the described functionality.

Keeping in mind the various devices and systems of FIG. 1, a flow chartdepicting acts for segmenting a tissue image using such devices andsystems is depicted in FIG. 2. In the depicted implementation, one ormore tissue images 50, such as images of a TMA, undergo featureextraction (Block 54) where color and texture values are obtained foreach pixel of the image 50 and represented as respective feature vectors56 for the image pixels. Examples of tissue images 50 are provided inFIG. 3. The vectors 56 of all the pixels in tissue image 50 arepre-processed (Block 58) to select a training subset 60 of pixels forunsupervised clustering. The training subset 60 of pixels may berandomly selected or selected based on image features such ashomogeneity. The training subset 60 is processed (Block 62) using atwo-dimensional self-organizing map (SOM). Processing the trainingsubset 60 using the two-dimensional SOM captures the dominant color andtexture features of the training subset 60 and then iterativelydetermines the dominant color and texture features of all pixel subsetswithin the tissue image 50. Based upon this training process, a set oftrained weights 66 is derived which represents pixel subsets in thetissue image 50 with updated values according to the dominant color andtexture features determined from the training subset 60. The tissueimage 50 is processed (Block 70) using a one-dimensional SOM and thetrained weights 66 such that pixels of the tissue image 50 are updatedbased upon the closest updated pixel values found in the trained weights66. The product of this updating HSOM process is an initially segmentedimage 72.

In some implementations, the initially segmented image isover-segmented. In such implementations, the initially segmented image72 may undergo a region merging process (Block 74) by which the numberof segments in the initially segmented image is reduced to arrive at aperceptually consistent segmentation in the form of a segmented image76. The steps involved in different region merging methods will befurther described below in the discussion of FIGS. 7 and 8. After regionmerging, the segmented image 76 will typically be more homogeneouswithin regions and have more disparity between the regions. In someimplementations, to further ease visualization, the segmented image 76may be recolored (Block 78) to produce a pseudo-colored and labeledsegmented tissue image 82. Examples of pseudo-colored segmented tissueimages 82 are provided in FIG. 4.

While the preceding provides a general overview of the presenttechnique, a more detailed discussion of certain aspects of the presenttechnique is discussed below. For example, a flow chart depictingfeature extraction and the training stage, is provided in FIG. 5.Turning now to FIG. 5, feature extraction begins with determining (Block104) the color components 112 and determining (Block 108) the texturecomponents 116 for each pixel in a tissue image 50. In oneimplementation, the color components of a pixel may include R, G, Bvalues, or other color values in accordance with another color standard.Determining the color component of each pixel (Block 104) provides pixelcolor components 112 for each pixel.

In an exemplary embodiment, the texture components are derived based onLaws' texture energy transforms where five simple one-dimensionalfilters (generally denoted as level, edge, spot, wave, and ripple) areconvolved with the transposes of each other to provide a set oftwo-dimensional symmetric and anti-symmetric center-weighted masks. Foursuch masks are E5′L5, E5′S5, R5′R5, and L5′S5, where the letterscorrespond with one of the five filters. These masks are convolved withthe tissue image 50 to produce a number of feature images that estimatethe energy within the pass-band of their associated filters. Thus, Laws'texture energy transforms may effectively discriminate among texturefields. After the tissue image 50 is convolved with each useful mask toobtain corresponding texture energy images, each pixel in the tissueimage 50 has a texture value for each of the masks it was convolvedwith, and these texture values are the respective pixel texturecomponents 116 for each pixel. The pixel color components 112 and pixeltexture components 116 are represented (Block 120) as a feature vector124 for each pixel so that each pixel has a corresponding feature vector124 which contain both color 112 and texture 116 components. The featurevector 124 of each pixel in the tissue image 50 form a matrix of featurevalues that is the input for unsupervised clustering.

From the matrix of feature vectors 124, a subset of pixels to be usedfor training is selected (Block 128) from the tissue image 50. Theselection process may be random or sequential based on image featuressuch as homogeneity or non-homogeneity. From this training subset 60, asample pixel is selected (Block 136). The selection process for thesample pixel 140 may be random or based on image features such ashomogeneity or non-homogeneity. The vector distances between the samplepixel 140 and the remaining pixels in the training subset 60 arecalculated to identify (Block 144) the pixel within the training subset60 with the closest vector distance to the sample pixel 140. This pixelis the best matching pixel 148 to that of the sample pixel 140. In oneexemplary embodiment, the best matching pixel 148 (denoted below asm_(b)) to the sample pixel 140 (denoted below as x), out of everyfeature vector (denoted below as m_(i)) in the training subset 60, maybe determined using the equation below:

$\begin{matrix}{{{x - m_{b}}} = {\min\limits_{i}{\left\{ {{x - m_{i}}} \right\}.}}} & (1)\end{matrix}$

The best matching pixel 148 and its topological neighbors are movedcloser (Block 152) to the input vector. In one exemplary embodiment, theprocess of moving the best matching pixel 148 and its topologicalneighbors closer to the input vector may be depicted in the equationbelow:

m _(i)(t+1)=m _(i)(t)+a(t)h _(bi)(t)[x−m _(i)(t)],  (2)

where t denotes time, a(t) is the learning rate and h_(bi)(t) is aneighborhood kernel centered on x, the sample pixel. In oneimplementation, the learning rate a(t) and the neighborhood radius ofh_(bi)(t) decrease monotonically with time. Because of the neighborhoodrelations, neighboring pixel values are pulled in the same direction asthe best matching pixel 148 towards the sample pixel 140, so that pixelvectors of neighboring units resemble each other. Once all the pixels inthe training subset 60 have been updated, as determined at decisionblock 156, and all the training subsets within the image have beenupdated (Block 160), the result is a coarsely segmented image 164 withall pixel subsets updated. In an exemplary embodiment, the weights ofthe pixels within the coarsely segmented image 164 tend to approximatethe density function of the vector inputs obtained from the image in anorderly manner.

Turning now to FIG. 6, a flow chart depicting acts for segmenting thetissue image 50 using the coarsely segmented image 164 is given. Thecoarsely segmented image 164 is a two-dimensional map, such as a 10×10map, that is utilized in the subsequent one-dimensional SOM, such as a1×20 SOM. Dominant features in the two-dimensional map depicted as acoarse segmented image 164 are represented in the one-dimensional array,and the most representative features are extracted from the tissue image50, resulting in its segmentation. To use the dominant features found inthe coarse segmented image 164, a pixel 172 from the original tissueimage 50 is selected (Block 168). The selection process for image pixels172 may be random or sequential or based on some feature characteristic.The feature vector of the presently selected image pixel 172 is analyzedand compared with the feature vectors of the coarse segmented image 164to identify (Block 176) the pixel 180 with the closest feature vector tothe image pixel 172. The image pixel 172 is then updated (Block 184) sothat its feature vector corresponds to that of the best matching pixel180. Once all the image pixels 172 have been updated, as determined atdecision block 188, an initial segmented image 72 is produced.

In certain exemplary embodiments, the initial segmented image 72 mayundergo region merging, as discussed in FIGS. 7 and 8. In particular,colors obtained at the end of the two-dimensional SOM may be consideredlabels assigned by the classification scheme. Thus, similar butdifferently labeled color regions in the initial segmented image 72correspond to the same useful tissue structures with similar initialfeature vectors. Hence, in a competitive learning implementation, suchsimilar color regions (represented as neurons or similar computationalelements) can be merged to arrive at a perceptually consistentsegmentation. In one embodiment, the merging is implemented using asuitable region-merging algorithm.

One such embodiment of region-merging using a threshold standard isdepicted in the flow chart of FIG. 7. In the depicted exemplarytechnique, each of the colors present in the initial segmented image 72represents a region. The color differences for each pair of colors arecalculated (Block 254), resulting in a list of color differences 258. Inone exemplary embodiment, the CIE (l*a*b*) color space may be used tocalculate color differences 258 according to the equation:

ΔE _(ab)=√{square root over ((Δl*)²+(Δa*)²+(Δb*)²)}{square root over((Δl*)²+(Δa*)²+(Δb*)²)}{square root over ((Δl*)²+(Δa*)²+(Δb*)²)}.  (3)

If the initial segmented image 72 had k colors, such as 20 colors wherethe initial segmented image 72 is the product of a 1×20 one-dimensionalSOM, then the equation above will result in a list of k(k−1)/2 colordifferences. A threshold 266 for region merging is then computed (Block262). In one embodiment, the threshold 266 (depicted below as T_(d)) maybe computed by subtracting the standard deviation (σ_(v)) from the mean(μ_(d)) of the list of color differences 258, as depicted in theequation below:

T _(d)=μ_(d)−σ_(v).  (4)

The threshold 266 may, in an alternative embodiment, be determined usingany other method to determine whether two color regions are close enoughto be merged. The depicted exemplary embodiment identifies regions withthe smallest color difference (Block 270) and these closest regions 274are compared (Block 278) with the threshold 266 to determine whethertheir color difference is within the threshold 266.

If the closest regions 274 do not have a color difference within thethreshold 266, then region merging is complete. In the depictedexemplary embodiment, upon completion of the region merging, theresulting segmented image 76 may be recolored and labeled according tothe recoloring (Block 282) to produce a pseudo-colored and labeledsegmented tissue image 82.

If the closest regions 274 do have a color difference within thethreshold 266, as determined at decision block 278, the closest regions274 are merged (Block 290) to generate an intermediate merged image 292.Region merging of the closest regions 274 results in a new color regionthat, in one embodiment, has an intermediate value between the closestregions 274. For example, in one implementation, the new color regionhas a color that is the median or mean of the two merged colors, oralternatively, the mode of the two merged colors. As depicted, theregions 274 of the intermediate merged image 292 with the smallest colordifference are identified (Block 270) and compared to the threshold 266and so forth until merging is completed, i.e., the closest regions 274are not within the threshold 266. In other words, in this exemplaryembodiment, the merging process will occur so long as there are regionpairs with color differences within the threshold 266.

FIG. 8 is a flow chart depicting acts for another embodiment of regionmerging using a region number target or goal. In this embodiment, thecolor differences 258 for each pair of color regions from the initialsegmented image 72 are calculated (Block 254). Of the resulting colordifferences 258, the regions with the smallest color difference areidentified (Block 270). These closest regions 274 are merged (Block 290)as described above. If, after this region merger, the number of regionsremaining in the resulting intermediate merged image 292 is equal to thenumber of color regions desired, as determined at decision block 296,the intermediate merged image 292 is the segmented image 76. Asdescribed above, in certain embodiments the segmented image 76 may berecolored and labeled (Block 282) to produce a pseudo-colored segmentedtissue image 82.

If after region merging (Block 290), the number of regions in theintermediate merged image 292 exceeds the target number of regions, theregions of the intermediate merged image 292 having the smallest colordifference are identified (Block 270) so that these closest regions 274can again be merged (Block 290). In this embodiment, region mergingcontinues until the number of regions remaining in the image equals thetarget number of regions. For example, in one embodiment, the targetnumber of regions may correspond to the number of useful or expectedtissue structures to be differentiated in the segmented image 76. In anexemplary embodiment, where the segmented image 76 is expected toinclude nuclei, stroma, epithelial tissue, and glands, the number ofuseful tissue structures, or the number of regions expected, would befour. Thus, region merging under this embodiment would cease once fourregions remained in the image. The target number of regions may also begreater than the number of useful tissue structures to be differentiatedin the segmented image 76, as the desired image may also presentdifferent colors for the same tissue structure. The number of targetregions may vary by application and may, for example, be two, three,four, five, six, and so forth up to the number of colors present in theinitially segmented image 72.

As will be appreciated, though the discussion of the two exemplarytechniques for region merging have been presented separately, bothtechniques may be employed in a single implementation. For example, insuch an exemplary implementation, the logic of both techniques may becombined such that the region merging process continues until no moreregions are within the threshold 266 or until the target number ofregions is reached.

While only certain features of the invention have been illustrated anddescribed herein, many modifications and changes will occur to thoseskilled in the art. It is, therefore, to be understood that the appendedclaims are intended to cover all such modifications and changes as fallwithin the true spirit of the invention.

1. A method for processing a color tissue image comprising: generating amatrix of feature vectors, wherein each feature vector corresponds to arespective pixel of a tissue image and describes color and texturefeatures of the respective pixel; selecting a subset of feature vectorsfrom the matrix of feature vectors; deriving a set of colors andtextures using the tissue image and the subset; and generating aninitial segmented tissue image using the set of colors and textures andthe tissue image.
 2. The method as recited in claim 1, comprisingextracting the two or more features for each pixel of interest in thetissue image.
 3. The method as recited in claim 1, wherein the color andtexture features comprise three color features and four texturefeatures.
 4. The method as recited in claim 1, wherein deriving the setof colors and textures comprises performing a two-dimensional selforganized mapping using the subset of feature vectors and the tissueimage.
 5. The method as recited in claim 1, wherein generating theinitial segmented tissue image comprises performing a one-dimensionalself organized mapping using the set of colors and textures and thetissue image.
 6. The method as recited in claim 1, comprising mergingregions within the initial segmented image to generate a segmented imagehaving fewer segments than the initial segmented image.
 7. The method asrecited in claim 6, comprising assigning colors or labels to thesegments of the segmented image to generate a pseudo-colored or labeledimage.
 8. The method as recited in claim 6, wherein merging regions isiteratively performed until a merge threshold criteria is no longersatisfied and/or until a target number of regions remain.
 9. The methodas recited in claim 1, comprising performing an automatic pathologyassessment of the initial segmented tissue image or an image derivedfrom the initial segmented tissue image.
 10. The method as recited inclaim 1, comprising establishing a training database using the initialsegmented tissue image or an image derived from the initial segmentedtissue image.
 11. The method as recited in claim 1, comprising traininga pathologist or a CAD algorithm using the initial segmented tissueimage or an image derived from the initial segmented tissue image. 12.One or more computer-readable media having a computer programcomprising: a routine configured to generate a matrix of featurevectors, wherein each feature vector corresponds to a respective pixelof a tissue image and describes color and texture features of therespective pixel; a routine configured to select a subset of featurevectors from the matrix of feature vectors; a routine configured toderive a set of colors and textures using the tissue image and thesubset; and a routine configured to generate an initial segmented tissueimage using the set of colors and textures and the tissue image.
 13. Theone or more computer-readable media as recited in claim 12, wherein thecomputer program comprises a routine configured to extract the two ormore features for each pixel of interest in the tissue image.
 14. Theone or more computer-readable media as recited in claim 12, wherein theroutine configured to derive the set of colors and textures performs atwo-dimensional self organized mapping using the subset of featurevectors and the tissue image.
 15. The one or more computer-readablemedia as recited in claim 12, wherein the routine configured to generatean initial segmented tissue image performs a one-dimensional selforganized mapping using the set of colors and textures and the tissueimage.
 16. The one or more computer-readable media as recited in claim12, wherein the computer program comprises a routine configured to mergeregions within the initial segmented image to generate a segmented imagehaving fewer segments than the initial segmented image.
 17. The one ormore computer-readable media as recited in claim 16, wherein thecomputer program comprises a routine configured to assign colors orlabels to the segments of the segmented image to generate apseudo-colored or labeled image.
 18. The one or more computer-readablemedia as recited in claim 16, wherein the routine configured to mergeregions iteratively merges regions until a merge threshold criteria isno longer satisfied and/or until a target number of regions remain. 19.An image analysis system, comprising: an imager configured to opticallyanalyze a tissue sample; data acquisition circuitry configured toacquire a tissue image of the tissue sample via the imager; and dataprocessing circuitry configured to generate a matrix of feature vectors,wherein each feature vector corresponds to a respective pixel of thetissue image and describes color and texture features of the respectivepixel, to select a subset of feature vectors from the matrix of featurevectors, to derive a set of colors and textures using the tissue imageand the subset, and to generate an initial segmented tissue image usingthe set of colors and textures and the tissue image.
 20. The imageanalysis system of claim 19, wherein the data processing circuitry isfurther configured to extract the two or more features for each pixel ofinterest in the tissue image.
 21. The image analysis system of claim 19,wherein the data processing circuitry is configured to derive the set ofcolors and textures by performing a two-dimensional self organizedmapping using the subset of feature vectors and the tissue image. 22.The image analysis system of claim 19, wherein the data processingcircuitry is configured to generate the initial segmented tissue imageby performing a one-dimensional self organized mapping using the set ofcolors and textures and the tissue image.
 23. The image analysis systemof claim 19, wherein the data processing circuitry is further configuredto merge regions within the initial segmented image to generate asegmented image having fewer segments than the initial segmented image.24. The image analysis system of claim 23, wherein the data processingcircuitry is configured to assign colors or labels to the segments ofthe segmented image to generate a pseudo-colored or labeled image. 25.The image analysis system of claim 23, wherein the data processingcircuitry is configured to merge regions iteratively until a mergethreshold criteria is no longer satisfied and/or until a target numberof regions remain.